Method and apparatus for controlling a camera module to compensate for the color spectrum of a white led

ABSTRACT

A method and an apparatus enabling use of a light source emitting a spectrum of light not following characteristics of a blackbody source of light with a camera module programmed for use with light sources following characteristics of blackbody sources of light, wherein a white-balance patch is transmitted to the camera module to cause the camera module to accept color coordinates representing a point on a chromaticity chart that partly defines a region of color coordinates on the chromaticity chart into which color coordinates of the light source fall and to which color coordinates of a reference white color derived by a white-balance routine of the camera module are constrained.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/723,059 filed Mar. 12, 2010, which is a continuation of U.S.application Ser. No. 11/626,886 filed Jan. 25, 2007. The contents ofU.S. application Ser. No. 12/723,059 and of U.S. application Ser. No.11/626,886 are incorporated herein by reference in their entirety.

BACKGROUND

1. Field

The disclosed and claimed concept relates generally to electronicdevices and, more particularly, to a method for controlling a cameramodule incorporated into a portable electronic device to compensate forthe characteristics of a white LED used as a flash for taking pictures.

2. Description of the Related Art

It is widely known to use a variety of different sources of light fortaking a picture with a digital camera module, including naturalsunlight, a xenon strobe, an incandescent bulb or a fluorescent bulb.Despite being very different light sources using very differentprocesses to emit light, a common characteristic of all of these lightsources is that the spectrums of light emitted by each of them, despitebeing different, provide a range of light frequencies that resemble theexpected behavior of radiant emissions of a blackbody at giventemperatures.

In 1931, an international committee called the Commission Internationalede L'Eclairage (CIE) met in Cambridge, England, and attempted to putforward a graphical depiction of the full range of colors of light thatthe human eye can actually perceive. This graphical depiction, namely achromaticity chart, and the resulting standard incorporating thischromaticity chart has come to be known as “CIE 1931” and is widely usedby scientists and photographers, among many others, in working withlight in the visible light spectrum. FIG. 1 depicts a simplifiedrepresentation of a chromaticity chart 100 based on the CIE 1931standard, with all visible colors of light specifiable with twodimensional color coordinates. As can be seen, towards the center ofwhat is frequently called the “horseshoe-shaped” visible region 110 ofall that the human eye can perceive is a white region 120 of colors oflight generally categorized as “white light” and surrounded by otherregions generally described as non-white light, including a red region121, a pink region 122 and a purple region 123. It should be noted thatthe exact boundaries of these regions 120-123 should be taken asapproximations and not precise designations of color, since theclassification of colors is necessarily subjective.

The human brain has evolved its own form of white-balancing capabilityby which human beings have little trouble discerning what color anobject should be, even though it may be illuminated with light that isonly marginally white, such as the reddish hue of the sun at sunset, theorange glow of campfire, or the bluish tint of a mercury vaporstreetlight. It is due to this flexibility of the human brain that anumber of light sources emitting a variety of different spectra oflight, and thereby having a variety of differing color coordinates thatoccupy different points on a chromaticity chart, can be classified as“white” light sources with the result that the white region 120 in FIG.1 occupies a considerable proportion of the visible region 110.

Passing through the white region 120 is a portion of a blackbody curve130 depicting the set of color coordinates of white light sources thatemit a spectrum of light frequencies that substantially follow thespectrum of light frequencies that would be expected to be emitted fromtheoretically ideal blackbody light sources heated to differenttemperatures. Most commonly used sources of white light have colorcoordinates specifying a point that falls along or substantially closeto this blackbody curve 130, including sunlight and xenon flash strobes,as well as incandescent, fluorescent, high-pressure sodium and mercuryvapor lamps. As a result of so many of the commonly used sources oflight used in taking pictures having color coordinates representingpoints that fall on or relatively close to the blackbody curve 130,algorithms, constants and limit values employed in digital cameras toperform automatic exposure control and automatic white-balancing arecommonly chosen and designed with a presumption that all light sourcesthat will be encountered will be ones with such color coordinates.Indeed, this presumption has become so ingrained that it has becomecommonplace for manufacturers of camera modules incorporated into otherelectronic devices to have such choices of algorithms, constants andlimit values built into or preprogrammed directly into the cameramodules, themselves.

As those skilled in the art of white-balancing algorithms willrecognize, a step taken by many known white-balancing algorithms isattempting to derive a reference white color in a given image as aninput parameter for determining the degree to which the colors in thatimage are to be adjusted to compensate for the lighting in the originalscene so that the objects in the resulting picture are presented withtheir correct colors. To do this, white-balancing algorithms typicallyrequire either that there be an object in the image that actually iswhite (known as the “white world” algorithm) or that the average of allthe colors of all the pixels in the image be a gray (known as the “grayworld” algorithm), and either of these approaches can provide a basisfrom which a reference white color for that image may be derived.However, it is possible to have images that do not provide a whiteobject or that are filled with objects of colors that provide a veryskewed result when averaging to derive a gray. An example of such animage is one filled with the tree leaves of a forest of trees such thatthe image is filled with different shades of green and little else,thereby providing no white objects and providing an average that willnecessarily be a green color and not a gray. If white-balancingalgorithms are allowed to process such an image without constraints, theresult can be whited-out or blackened-out objects in the resultingpicture, and so it is deemed desirable to specify boundaries for what areference white color may be so as to constrain the degree to which awhite-balancing algorithm is permitted to adjust colors.

Given the aforementioned presumption that the light sources to beencountered by a digital camera are likely to have color coordinatesspecifying points falling along or quite close to the blackbody curve130, the format in which the boundaries for what a reference white colormay be are communicated to typical camera modules in a manner thatcomports with this assumption. In this commonly used format, a pair ofcolor coordinates that define the endpoints of a straight segment in achromaticity chart, such as a segment 140 depicted in FIG. 1, arecommunicated to a camera module along with an error term (or “locus”)specified in terms of a maximum perpendicular distance away from thesegment 140. These two endpoints and the error term, together, specify arectangular-shaped reference white region 141 within the white region110 that defines these boundaries, thereby defining a set of acceptablecolor coordinates within which the white-balancing algorithm ispermitted to choose a color to be a reference white for a given image.This is to allow a short segment that should resemble a small portion ofthe blackbody curve 130 to be specified, such as segment 140, and thisshort segment should be positioned to either largely overlie a portionof the blackbody curve 130 or to be relatively close to and relativelyparallel with a portion of the blackbody curve 130. No allowance is madein this format for specifying the boundaries of a possible referencewhite with a reference white region having any other shape than arectangular region, such as the reference white region 141 shown.

Also, given the same aforementioned presumption that the light sourcesto be encountered by a digital camera are likely to have colorcoordinates specifying points falling along or quite close to theblackbody curve 130, it is commonplace to in some way build minimumand/or maximum limits on values used to define the reference whiteregion 141 such that values defining a reference white region 141 thatdoes not substantially overlie the blackbody curve 130, or that is notat least substantially close to the blackbody curve 130 are rejected.The effective result is to create a limit region, such as limit region145 depicted in FIG. 1, into which at least a portion of the whiteregion 141 must fall.

Of those light sources having color coordinates representing pointsfalling along or close to the blackbody curve 130, xenon strobes havebecome commonplace for use as flashes in portable electronic devicesused in photography. A xenon strobe is very small in size whileproducing an extremely bright light that very quickly illuminates asetting of which a picture is to be taken. The amount of illuminationneeded from a flash to sufficiently light a scene for scanning its imageis a measurable quantity and can be roughly calculated as the brightnessof the flash multiplied by the amount of time it must be turned on. Thebrighter the light source used as a flash, the less time it needs to beturned on to sufficiently light a scene. Furthermore, the amount of timethat a given flash needs to be turned is not necessarily related to theamount of time needed for an image scanning element (such as a CCDsemiconductor device or a CMOS imaging device) to actually scan an imageas part of the process of capturing that image. In other words, where abright flash is used, it is not unheard of to actually turn off theflash before the image scanning element has completed scanning theimage, because a sufficient amount of illumination has been supplied andleaving the flash on any longer would result in too high an amount ofillumination and portions of the image being whited out. However, wherea dimmer light source is used as a flash, the flash must be turned onfor a longer period of time to achieve the same amount of illuminationas a brighter light source, and it is often necessary to delay the startof scanning an image until a high enough amount of illumination has beenachieved.

Recently, a new artificial source of white light, the so-called whiteLED, has been introduced, providing the opportunity to create a flashfor use in digital photography that requires less power than other lightsources. Unfortunately, the white LEDs have a range of color coordinatesspecifying a range of points that fall substantially distant from theblackbody curve 130, and furthermore, at least partly fall outside thewhite region 120 and into the pink region 122. This deviation of whiteLEDs from the blackbody curve 130 is largely due to the manner in whichwhite LEDs produce light. White LEDs are in truth, blue LEDs that arepartially covered with a yellowish phosphor that converts part of theblue light into yellow light. The result is a blending of blue andyellow light frequencies that approximates white light well enough forthe human eye and the human brain to accept it as a source of whitelight. In essence, two different non-white light emissions, each havingits own spectrum of light frequencies, are being blended to approximatewhite light and such a mixing of two non-white spectra is notcharacteristic of blackbody sources of radiant energy.

Also, white LEDs, though brighter than incandescent lamps of comparablesize, are far dimmer than xenon strobes of comparable size. As a result,to achieve a desirable amount of illumination of a scene when used as aflash, a white LED must be kept on far longer than a xenon strobe usedas a flash, and a white LED must also be supplied with a very highamount of electric power that would actually damage internal componentsof the white LED if that amount of power were maintained for more than avery brief period of time. In using a white LED as a flash, the amountof time during which the white LED is actually turned on can be keptshort enough to prevent this damage. Unfortunately, even during thebrief period in which the white LED is turned on, the light emittingsemiconductor components of the white LED respond to the very highamount of power by converting an ever increasing proportion of thatpower into heat as time passes from the moment at which that power isfirst supplied to the moment when that power is removed.Correspondingly, as time passes the proportion of that power convertedto visible light decreases such that the white LED is initially verybright when that power is first applied, but that brightness levelalmost immediately begins fading more and more as time passes. With thisquickly fading of brightness, the color spectrum output by a white LEDalso changes quickly as time passes from the moment that it is turnedon. This changing light level and this changing color spectrum must betaken into account in both calculating the amount of time a white LED isto be turned on to provide a sufficient total amount of illumination toserve as an effective flash and in compensating for its changingspectrum of light output in performing white-balancing.

Another feature of white LEDs not exhibited by artificial light sourceslong used in photography, including xenon strobes and incandescentbulbs, is the high variability in the color spectra of each of the blueand yellow elements of the light emitted by white LEDs. White LEDs andthe technology to manufacture them are still sufficiently new that onlyslow progress has been made in exerting tighter control over themanufacture of white LEDs to achieve sufficient consistency to avoidhaving two white LEDs from the very same production run emit light thatis of perceptibly different hues. For this reason, unlike otherartificial light sources that have far higher consistency in the spectraof their emitted light, the size of the region of that the “white” lightemitted by LEDs may fall within is considerably larger than for otherlight sources. As a result of these various issues, current practices incontrolling a camera module's built-in white-balancing algorithm areinsufficient to accommodate the very unique characteristics of whiteLEDs.

BRIEF DESCRIPTION OF THE DRAWINGS

A full understanding of the disclosed and claimed concept can be gainedfrom the following Description when read in conjunction with theaccompanying drawings in which:

FIG. 1 is a simplified depiction of a CIE 1931 chromaticity chartdepicting the black body curve and a PRIOR ART approach to specifying areference white region within which a reference white color isconstrained;

FIG. 2 is a depiction of an improved handheld electronic device inaccordance with the disclosed and claimed concept;

FIG. 3 is a schematic depiction of the improved handheld electronicdevice of FIG. 2;

FIG. 4 is another simplified depiction of a CIE 1931 chromaticity chartdepicting the black body curve and an approach to specifying a referencewhite region within which a reference white color is constrained inaccordance with the disclosed and claimed concept; and

FIG. 5 is a flowchart depicting an embodiment of an improved method inaccordance with the disclosed and claimed concept.

DESCRIPTION

The accompanying figures and the description that follows set forth thedisclosed and claimed concept in its preferred embodiments. It is,however, contemplated that persons generally familiar with handheldelectronic devices will be able to apply the novel characteristics ofthe structures and methods illustrated and described herein in othercontexts by modification of certain details. Accordingly, the figuresand description are not to be taken as restrictive on the scope of thedisclosed and claimed concept, but are to be understood as broad andgeneral teachings.

For purposes of the description hereinafter, the terms “upper”, “lower”,“right”, “left”, “vertical”, “horizontal”, “top”, “bottom”, andderivatives thereof shall relate to the disclosed and claimed concept asit is oriented in the figures.

An improved electronic device 200 is depicted generally in FIG. 2 and isdepicted schematically in FIG. 3. The electronic device 200 may be ahandheld or other portable electronic device (e.g. and withoutlimitation, a digital camera, a PDA, a cell phone, a digital watch, or alaptop computer). The electronic device 200 incorporates a housing 202on which are disposed a white LED 207 serving as a flash for takingpictures and a camera module 250. The housing 202 may additionally havedisposed thereon an input device 204 and/or an output device 205. Theelectronic device 200 also incorporates a processor 210 connected to astorage 220, and a LED interface 237 controlling the LED 207. Theprocessor 210 may additionally be connected to one or more of an inputinterface 234 receiving input from the input device 204, an outputinterface providing output to the output device 205, and a media storagedevice 240 capable of interacting with a storage medium 241 (which mayor may not be of removable form). The camera module 250 incorporates aprocessor 260 connected to a storage 270, an exposure control element282 and an image scanning element 283. Although described and depictedas being disposed on the housing 202 of the electronic device 200, thewhite LED 207 and/or the camera module 250 may alternatively bephysically separate from the housing 202, but linked to other componentsof the electronic device 200 through a suitable electrical, optical,radio frequency or other linkage.

The processors 210 and 260 may be of any of a wide variety of processingdevices, including and without limitation, microcontrollers,microprocessors, sequencers, digital signal processors or state machinesimplemented in hardware logic. In some embodiments, one or both of theprocessors 210 and 260 may be one of a number of commercially availableprocessors executing at least a portion of the widely known and used“X86” instruction set and/or another instruction set.

The media device 240 and the storages 220 and 270 may be of any of awide variety of types of storage devices, including and withoutlimitation, disk drives (e.g. and without limitation, hard drives,floppy drives, magneto-optical drives, magnetic tape drives or CD-ROMdrives), solid state memory (e.g. and without limitation, static RAM,dynamic RAM, ROM, EEPROM or FLASH) and memory card readers. However, inpreferred practice, the storages 220 and 270 are generally more capableof supporting speedy random accesses than the media device 240, and themedia device 240 is capable of supporting removable media while thestorages 220 and 270 are not meant to be removable. In preferredpractice, it is generally intended that the removable media device 240support the exchange of data and/or software between the electronicdevice 200 and another electronic device (not shown) through the storagemedium 241.

The white LED 207 may be any of a variety of semiconductor-based lightemitting diodes capable of emitting light that substantiallyapproximates white light. The white LED 207 may be fabricated byapplying a coating to a blue LED that converts at least some of theemitted blue light into a yellow light such that a combination of blueand yellow light is produced that approximates white light to theperception of the human eye. Alternatively, the white LED 207 may befabricated in other ways as those skilled in the art will readilyrecognize, including, but not limited to, adding or applying red andgreen phosphors to a blue LED. The LED interface 237 allows theprocessor 210 to control when the white LED 207 is turned on and mayallow the processor 210 to control the intensity of the light emitted bythe white LED 207.

The camera module 250 may be any of a variety of commercially availablecamera modules fabricated by a variety of manufacturers for the purposeof being incorporated into other devices, such as the electronic device200. The image scanning element 283 may be one of a variety of availablecharge-coupled devices (CCD) or CMOS imaging devices, or may be anothersuitable form of device capable of scanning an image of objects in itsview. The exposure control element 282 provides an aperture ofcontrollable dimensions through which the light from the objects in theview of the image scanning element 283 passes to reach the imagescanning element 283. Alternatively, the exposure control element 282may control the amount of light reaching the image scanning element 283in other ways known to those skilled in the art.

The input device 204 may be of any of a variety of input devices capableof accepting input from a user of the electronic device 200, includingwithout limitation switches, a keypad, a joystick, a rollerball, or atouchpad. In embodiments that incorporate the input device 204, theinput interface 234 couples the processor 210 to the input device 204 toreceive input therefrom. The output device 205 may be of any of avariety of output devices capable of providing information to a user ofthe electronic device 200, including without limitation lights, adisplay device, an audible indicator, or a tactile device such as avibrator mechanism causing the electronic device 200 to vibrate suchthat a user of the electronic device 200 is able to feel the vibration.In embodiments that incorporate the output device 205, the outputinterface 235 couples the processor 210 to the input device 205 toprovide output thereto.

When the electronic device 200 is used to take a picture, the processor210 accesses the storage 220 to retrieve and execute a sequence ofinstructions of a control program 222, thereby causing the processor 210to transmit sequences of instructions and/or data to the camera module250 and to operate the camera module to scan one or more images as willshortly be explained. In turn, the processor 260 accesses the storage270 to retrieve and execute sequences of instructions from awhite-balance routine 272, an exposure routine 274 and/or anothersequence of instructions provided by the processor 210, thereby causingthe processor 260 to operate the exposure control element 282 and theimage scanning element 283 to carry out the scanning of one or moreimages. The processors 210 and 260 are caused to interact to transferthe data representing the resulting picture from the camera module 250to be stored in the storage 220, or perhaps the media storage device 240if present. Where the taking of a picture entails the use of the whiteLED 207 as a flash, the processors 210 and 260 may be caused to furtherinteract in controlling the timing and intensity of the lightingsupplied by the white LED 207 through the LED interface 237.

In embodiments of the electronic device 200 having the input device 204and/or the output device 205, the processor 210 is further caused by thecontrol program 222 to operate the input interface 234 and/or the outputinterface 235 to interact with the user of the electronic device 200through one or both of the input device 204 and the output device 205.Where the input device 204 includes a relatively small number ofswitches providing the user with the ability to control various aspectsof the process of taking a picture (e.g. without limitation, the focus,the landscape or portrait mode, and whether or not to use a flash), theprocessor 210 receives such input from the user and carries out thetaking of a picture, accordingly. Where the input device 204 includes akeypad or other device providing greater flexibility of input, the usermay be provided with the ability to enter data concerning the picture tobe taken, such as a time, place or name of the subject of the picture.Where the output device 205 includes a graphical display, the processor210 may be caused by the control program 222 to present the user with aview of what the image scanning element 283 sees before the picture istaken and/or a view of the resulting picture on the output device 205.

In embodiments of the electronic device 200 having the media storagedevice 240, the processor 210 may be further caused to store picturestaken by the user on the storage medium 241 for the user to transfer toanother electronic device (not shown) for display, archiving and/orprinting. Where such embodiments also incorporate a form of both theinput device 204 and the output device 205 of sufficient ability, theprocessor 210 may be further caused to provide the user of theelectronic device 200 with the ability to use the input device 204 andthe output device 205 to view and select pictures to be stored on thestorage medium 241, as well as to select pictures to be deleted.Alternatively, or in addition to the media storage device 240, theelectronic device 200 may further incorporate a communications interface(not shown) allowing the electronic device 200 to be directly connectedto another electronic device for the transferring of pictures, otherdata and/or software (e.g. without limitation, a digital serialinterface such as IEEE 1394).

As previously described, the camera module 250 may be any one of avariety of commercially available camera modules from a variety ofmanufacturers for incorporation into various electronic devices,including the electronic device 200. The white-balance routine 272 maybe based on any of a variety of widely known white-balancing algorithms(including the earlier-described gray world and white world algorithms)to derive a reference white color for a given image that is used todetermine the degree to which the white-balance routine 272 is to beused to modify that image to compensate for the lighting used. However,as was also previously described, it is common practice to imposeconstraints on white-balancing algorithms to prevent overcompensationfor lighting that can result where the colors in an image do not providewhite-balancing algorithms with the reference colors needed to function,correctly.

Unfortunately, the commonplace manner of describing the reference whiteregion of color coordinates to which the point representing a referencewhite color is to be constrained as a rectangular region that issubstantially adjacent to or that substantially overlies a portion of ablackbody curve on a chromaticity chart is based on the assumption thatwhatever source of light is used to illuminate an image will exhibitcharacteristics largely conforming to what would be expected of acorresponding blackbody source of radiation. This same assumption hasalso resulted in the commonplace practice of incorporating into aparameter data 273 within the storage 270 a set of minimum and maximumvalue limits that will be accepted for specifying the color coordinatesdefining the segment that partly defines that rectangular region. Ineffect, these minimum and maximum value limits describe a limit regioninto which at least a portion of the white reference region must fall.Unfortunately, to describe a rectangular region within which the colorcoordinates of the white LED 207 are likely to fall requires specifyingcolor coordinates that are outside such minimum and maximum valuelimits.

As part of the earlier-described process of taking a picture where thewhite LED 207 is employed as a flash, the processor 210 is caused by thecontrol program 222 to provide the camera module 250 with a pair ofcolor coordinates (i.e., the pair of points defining a segment) and anerror term (or “locus”) that define a rectangularly-shaped referencewhite region to which the point defined by the color coordinates of thereference white color derived by the white-balance routine 272 are to beconstrained. However, to overcome the commonplace limitations imposed bythe minimum and maximum value limits stored within the parameter data273, the processor 210 is first caused by the control program 222 totransmit to the camera module 250 a white-balance patch 225 retrieved bythe processor 210 from the storage 220. In some embodiments, thewhite-balance patch 225 provides at least one alternate minimum and/ormaximum value that the processor 260 uses in place of at least oneminimum and/or maximum value of the parameter data 273 when executing asequence of instructions of the white-balance routine 272. In otherembodiments, the white-balance patch 225 provides an alternate sequenceof instructions to be executed by the processor 260 in place of at leasta portion of a sequence of instructions of the white-balance routine272. In still other embodiments, both alternate value(s) and alternateinstructions are provided. In effect, these alternate value(s) and/orinstructions redefine the boundaries of the limit region into which atleast a portion of a reference white region must fall. This redefiningmay entail resizing the limit region, may entail shifting the positionof the limit region on a chromaticity chart, or may entail both.

FIG. 4 depicts a chromaticity chart 150, that when compared to thechromaticity chart 100 of FIG. 1, illustrates the change enabled withtransmission of the white-balance patch 225 to the camera module 250.Upon receiving the white-balance patch 225, the limit region 145 ofchromaticity chart 100 is redefined to create an alternate limit region195 such that the camera module 250 is able to accept a pair of colorcoordinates defining a pair of points that define an alternate segment190, that along with an appropriate error term, define arectangularly-shaped alternate reference white region 191. The alternatereference white region 191 defines the constraints to which colorcoordinates specifying a point for a reference white color that bettercorresponds with the use of the white LED 207 will be held. As can beseen, unlike the reference white region 141, at least a portion of thealternate reference white region 191 lies outside the limit region 145and the alternate reference white region 191 neither overlies nor issubstantially adjacent to the blackbody curve 130.

It should be noted that although the alternate limit region 195 isdepicted as a tilted rectangular region not unlike the reference whiteregion 191, other configurations of the limit region 195 are possible asthose skilled in the art will readily recognize. Furthermore, in variousembodiments, the alternate limit region 195 may represent an expansionof the limit region 145, a shifting of the limit region 145, areplacement of the limit region 145, a provision of an alternate limitregion in addition to the limit region 145, or other form ofredefinition of the limit region 145 as those skilled in the art willreadily recognize.

As also previously discussed, the white LED 207 is a dimmer source ofvisible light than a xenon strobe, requiring that the white LED 207remain turned on longer than a xenon strobe to achieve a comparableamount of illumination in illuminating a scene. However, even thoughthis longer period is still short enough to be perceived by the humaneye as a mere flash, it is long enough for the level of visible lightoutput by the white LED 207 to fade significantly from the time thewhite LED 207 is first turned to the time the white LED 207 is turnedoff. This changing light level adds complexity to the operation of theexposure routine 274 that causes the processor 260 to control theexposure control element 282 to adjust the amount of light received bythe image scanning element 283 in taking a picture. Also, as thoseskilled in the art will readily recognize, a changing light level from alight source also results in a changing color spectrum for the lightoutput by that light source, and this adds complexity to the operationof the white-balance routine 272. To address the changing light leveland the changing color spectrum occurring during use of the white LED207 as a flash, the processor 210 is caused by the control program 222to operate the camera module 250 in a mode that is normally reserved foruse in lighting conditions in which no flash is used.

In the prior art digital cameras, it is commonplace to operate a cameramodule to scan a single image when a flash is used, and to operate acamera module to scan a succession of images when a flash is not used.With light sources that might be used as a flash, other than a whiteLED, the characteristics of the spectrum of light emitted are moretightly controllable, are more consistent and are far better known giventhe many years that other light sources have been used in photography.Therefore, a camera module may be given a set of highly reliable initialvalues as inputs to white-balancing and/or exposure algorithms withconsiderable confidence that those values are likely to be correctenough for most forms of scenery such that an image of that scenery needbe scanned only once. However, where a flash is not used, the lightingwithin the scene, itself, is being relied upon to provide the necessaryillumination, and unlike a flash incorporated into a digital camera, thecharacteristics of that lighting cannot be described to the cameramodule with such precision. Therefore, it is commonplace to supply thecamera module with a highly general set of initial values that aredeemed most likely to be applicable to most situations, and to then scana rapid succession of images when the user of the digital camera pressesa button to take a picture. With each successive image, the initialvalues are adjusted based on an analysis of the results of applying theinitial values to the preceding image.

It should be noted that part of the reason that the scanning of multipleimages is necessary to support the successive use of white-balancingand/or exposure algorithms is that it is commonplace practice to keepthe costs of camera modules low by not incorporating a frame buffer intocamera modules. In other words, white-balancing and exposure algorithmsmust be applied to the pixels of an image as the image seen by a cameramodule's image scanning element is scanned by the image scanningelement. If a frame buffer capable of holding an entire scanned imagewere incorporated into a camera module, it would be possible to actuallycapture an image (wherein capturing an image entails both scanning andstoring the image, and not merely scanning it) and repeatedly applywhite-balancing and/or exposure algorithms to the single captured imagewhile refining the initial values with each application of thosealgorithms. The scanning of the 3, 4, 5 or possibly more images occursquickly enough that the user of the digital camera does not realize thatmultiple images are being scanned in taking a single picture, but withthe scanning of a succession of images, the white-balancing and/orexposure algorithms are provided with multiple opportunities to moreprecisely compensate for unknown lighting conditions.

As part of the earlier-described process of taking a picture where thewhite LED 207 is employed as a flash, the processor 210 is caused by thecontrol program 222 to operate the camera module 250 as if a picturewithout a flash were being taken such that the camera module 250 iscaused to scan a succession of images. The processor 210 is caused tosupply the camera module 250 with initial values to be employed by theprocessor 260 in executing sequences of instructions from thewhite-balance routine 272 and/or the exposure routine 274. Unlike asituation in which lighting with unknown characteristics at a scene isemployed, the characteristics of the white LED 207 are known, althoughsome of those characteristics are expected to vary. Also, the processor210 may be caused to perform one or more calculations to derive one ormore of the initial values based on one or more inputs, such as a manualsetting regarding light levels or exposure time provided by a user ofthe electronic device 200, or an input from a light level sensor (notshown).

The processor 210 then directs the camera module 250 to begin scanning asuccession of images of a scene and the processor 210 operates the LEDinterface 237 to turn on the white LED 207. Use is made of the fact thatthe white LED 207 is sufficiently dim that the white LED 207 will haveto be turned on for a period of time long enough that a number of imagescan easily be scanned by the camera module 250 while the LED 207 isstill turned on. The scanning of images is directed by the processor 210to begin within a very short period of time after the white LED 207 isturned on. Although this results in a relatively small amount ofillumination having been applied to a scene at the time the first imageis scanned, such an initial amount of lighting is sufficient for theprocessor 260 to execute sequences of instructions of the white-balanceroutine 272 and/or the exposure routine 274 to begin the process ofrefining the initial values to compensate for the light provided by thewhite LED 207. As each successive image is scanned, more time passesduring which the white LED 207 is turned on and more of the totalillumination required to fully illuminate a given scene is supplied bythe white LED 207.

During the scanning of these successive images, the spectrum of lightprovided by the white LED 207 changes as an increasing proportion of theelectrical energy supplied to the white LED 207 is converted to heat(instead of visible light) over time. However, the degree of change inthe spectrum between the scans of successive images is small enough thatthe process of successive refinement of the initial values is able tocompensate for it. In some embodiments, the quantity of successiveimages to be scanned is a set value preprogrammed into the camera module250. In other embodiments, the degree to which the initial values arechanged with each refinement accompanying the scanning of eachsuccessive image is analyzed to determine if the degree of change hasreached a low enough threshold level between the two most recentlyscanned images that there is unlikely to be further significantrefinement with the scanning of any more images.

FIG. 5 is a flow chart of an embodiment of an electronic device with awhite LED and a camera module being used to take a picture. Starting at510, the electronic device awaits an indication from a user of theelectronic device to take a picture. Such an indication may be suppliedby the user pressing a button disposed on the housing of the electronicdevice, a timer set by the user, or another mechanism under the user'scontrol. At 520, the next step is determined by whether or not the whiteLED of the electronic device is used as a flash in taking the picture.Whether or not the white LED is used may be determined by an inputprovided by the user (e.g., without limitation, a switch operated by theuser to control use of the white LED) or by an automated function of theelectronic device in which the white LED is employed as a flash if asensor of the electronic device detects insufficient light to take thepicture without using the white LED.

If at 520, the white LED is not used as a flash, then a set of generalinitial values applicable to a wide variety of possible light sourcesthat may be present in a given scene are transmitted to the cameramodule at 530. Given that natural sunlight is highly likely to be thesource of light relied upon in such circumstances, the initial valuesare likely to be chosen to prepare the light-balancing and/or exposureroutines of the camera module for a light source having characteristicsconsistent with a blackbody source of light. One or more of the initialvalues may be derived and/or modified by a processor of the electronicdevice into which the camera module is incorporated to take into accountinput from a light level sensor providing an indication of the amount oflight available in the given scene.

At 532, the electronic device signals the camera module to scan asuccession of images. As discussed earlier, repetitive application ofwhite-balancing and/or exposure routines allows greater refinement ofthe initial values, and the scanning of a succession of images of agiven scene is necessary for a camera module having no frame buffer inwhich a single complete scanned image could be stored (such that theimage could be said to have been “captured” rather than simply scanned)to allow the white-balancing and/or exposure routines to be repeatedlyapplied to a single captured image.

However, if at 520, the white LED is used as a flash, then awhite-balance patch is transmitted to the camera module at 540. Aspreviously discussed, it is common practice for camera modules to beprogrammed with minimum and maximum limits for initial valuestransmitted to the camera module by the electronic device that are basedon the assumption that all light sources will exhibit characteristicsconsistent with a blackbody source of light. These initial valuesinclude values defining a reference white region on a chromaticity chartinto which the point specified by the color coordinates of a referencewhite color derived by the camera module's white-balancing routine mustfall. However, as previously discussed, the typical minimum and maximumlimits for these initial values define a limit region that does notpermit color coordinates defining a reference white region at a locationmore appropriate for a white LED to be specified. The white-balancepatch transmitted at 540 provides the camera module with at least onealternate minimum/maximum value more appropriate for those colorcoordinates and/or provides a substitute sequence of instructions to beexecuted by a processor within the camera module in place of at least aportion of the camera module's white-balancing routine.

At 542, initial values corresponding to the use of the white LED aretransmitted to the camera module, including a pair of segments and anerror term defining a reference white region into which the pointdenoted by the color coordinates of a derived reference white color willbe constrained. At 544, the electronic device signals the camera moduleto scan a succession of images. During the scanning of those images, theinitial values corresponding to the use of the white LED are furtherrefined to allow more precise compensation for the changing lightprovided by the white LED. At 546, with the picture having been taken,the electronic device may transmit a signal to the camera module causingthe erasure of the white-balance patch from a storage within the cameramodule, or causing some other action to occur that counteracts thewhite-balance patch.

While specific embodiments of the disclosed and claimed concept havebeen described in detail, it will be appreciated by those skilled in theart that various modifications and alternatives to those details couldbe developed in light of the overall teachings of the disclosure.Accordingly, the particular arrangements disclosed are meant to beillustrative only and not limiting as to the scope of the disclosed andclaimed concept which is to be given the full breadth of the claimsappended and any and all equivalents thereof.

1. An electronic device comprising: a flash source; a camera module withan image scanning element and a pre-determined white-balance routinethat includes an initial limit region of a chromaticity chart, theinitial limit region of the chromaticity chart overlying a portion of ablackbody radiation curve, the flash source having an optical spectrumlying outside the initial limit region of the chromaticity chart, thecamera module being operable in an ambient light mode in which the imagescanning element is controlled to capture and white-balance a successionof images, and in an artificial light source mode in which the imagescanning element is controlled to capture and white-balance a singleimage; a storage containing a white-balance patch and color coordinatesof points on the chromaticity chart; and a processor operable toretrieve the white-balance patch from the storage and to provide thewhite-balance patch to the camera module, the camera module beingoperable to modify the initial limit region of the white-balance routinein accordance with the white-balance patch to obtain a modified limitregion, the modified limit region including at least a portion of theflash source optical spectrum, the processor also being operable toretrieve the color coordinates of a first point of the chromaticitychart and to provide the color coordinates of the first point to thecamera module, the first point being outside the initial limit regionand inside the modified limit region, the first point to be used by thewhite-balance routine to define a reference white region, the referencewhite region excluding any portion of the black body radiation curve,the processor being further operable to energize the flash source and tosignal the camera module to operate in the ambient light mode.
 2. Theelectronic device of claim 1 further comprising a sensor to detect anamount of light, the sensor being operationally connected to theprocessor, the processor to determine, in accordance with a signalprovided by the sensor, that the amount of light is insufficient to takea picture and to cause the flash source to be energized and the cameramodule to operate in the ambient light mode.
 3. The electronic device ofclaim 2 further comprising an exposure control element to control aquantity of light impinging on the image scanning element, the cameramodule having an exposure routine, the camera module to operate theexposure control element in accordance with the exposure routine.
 4. Theelectronic device of claim 1 wherein the flash source is a lightemitting diode and the image scanning element is one of a charge coupleddevice and a CMOS imaging device.
 5. A method of taking a picture withan electronic device, the electronic device having a storage containinga white balance patch, a flash source, and a camera module, the cameramodule having an image scanning element and a pre-determinedwhite-balance routine that includes an initial limit region of achromaticity chart, the initial limit region of the chromaticity chartoverlying a portion of a blackbody radiation curve, the flash sourcehaving an optical spectrum lying outside the initial limit region of thechromaticity chart, the camera module being operable in an ambient lightmode in which the image scanning element is controlled to capture andwhite-balance a succession of images, and in an artificial light sourcemode in which the image scanning element is controlled to capture andwhite-balance a single image, the method comprising: providing thewhite-balance patch to the camera module; modifying the initial limitregion of the white-balance routine in accordance with the white-balancepatch to obtain a modified limit region, the modified limit regionincluding at least a portion of the flash source optical spectrum;receiving at the camera module, color coordinates of a first point ofthe chromaticity chart, the first point being outside the initial limitregion and inside the modified limit region, the first point being usedby the white-balance routine to define a reference white region, thereference white region excluding any portion of the black body radiationcurve; energizing the flash source; and operating the camera module inthe ambient light mode.
 6. The method of claim 5 wherein providing thewhite-balance patch to the camera module is preceded by retrieving thewhite-balance patch from the storage.
 7. The method of claim 6 whereinretrieving the white-balance patch from the storage is preceded byreceiving an indication to take the picture.
 8. The method of claim 5wherein the electronic device further has a sensor to detect an amountof light, wherein energizing the flash source is preceded by the sensordetecting that the amount of light is insufficient to take a picture. 9.The method of claim 5 wherein energizing the flash source is preceded byreceiving color coordinates of a second point of the chromaticity chart,the second point being outside the initial limit region and inside themodified limit region, the second point being used by the white-balanceroutine to further define the reference white region.
 10. The method ofclaim 9 wherein the first and second points are endpoints of a segmentof the reference white region.
 11. The method of claim 10 furthercomprising receiving a locus, the locus being used by the white-balanceroutine to further define the reference white region.
 12. The method ofclaim 11 wherein the locus specifies a perpendicular distance away fromthe segment.
 13. The method of claim 12 wherein the reference whiteregion is a rectangular reference white region.
 14. A method of making acamera module compatible with a flash source, the camera module having apre-determined white-balance routine that includes an initial limitregion of a chromaticity chart, the initial limit region of thechromaticity chart overlying a portion of a blackbody radiation curve,the flash source having an optical spectrum lying outside the initiallimit region of the chromaticity chart, the camera module being operablein an ambient light mode in which the image scanning element iscontrolled to capture and white-balance a succession of images, and inan artificial light source mode in which the image scanning element iscontrolled to capture and white-balance a single image, the methodcomprising, at the camera module: modifying the initial limit region ofthe white-balance routine in accordance with at least one input value toobtain a modified limit region, the modified limit region including atleast a portion of the flash source optical spectrum; receiving colorcoordinates of a first point of the chromaticity chart, the first pointbeing outside the initial limit region and inside the modified limitregion, the first point being used by the white-balance routine todefine a reference white region, the reference white region excludingany portion of the black body radiation curve; and receiving anindication to operate in the ambient light mode.
 15. The method of claim14 wherein the step of modifying the initial limit region is preceded byreceiving the at least one input value.
 16. The method of claim 14further comprising receiving color coordinates of a second point of thechromaticity chart, the second point being outside the initial limitregion and inside the modified limit region, the second point being usedby the white-balance routine to further define the reference whiteregion.
 17. The method of claim 16 wherein the first and second pointsare endpoints of a segment of the reference white region.
 18. The methodof claim 17 further comprising receiving a locus, the locus being usedby the white-balance routine to further define the reference whiteregion.
 19. The method of claim 18 wherein the locus specifies aperpendicular distance away from the segment.
 20. The method of claim 19wherein the reference white region is a rectangular reference whiteregion.